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1 Introduction 

In this paper we study asymptotic behavior of regular subsets in a free group 
F of finite rank, compare their sizes at infinity, and develop techniques to 
compute the probabilities of sets relative to distributions on F that come 
naturally from random walks on the Cayley graph of F. We apply these 
techniques to study cosets, double cosets, and Schreier representatives of 
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finitely generated subgroups of F with an eye on complexity of algorithmic 
problems in free products with amalgamation and HNN extensions of groups. 

During the last decade it has been realized that a natural set of algebraic 
objects usually can be divided into two parts. The large one (the regular 
part) consists of typical, "generic" objects; and the smaller one (the "singu- 
lar" part) is made of "exceptions" . Essentially, this idea appeared first in the 
form of zero-one laws in probability theory, number theory, and combina- 
torics. It became popular after seminal works of Erdos, that shaped up the 
so-called Probabilistic Method (see, for example, [1]). In finite group theory 
the idea of genericity can be traced down to a series of papers by Erdos and 
Turan in 1960- 70's (for recent results see, for example, [55]). In combina- 
torial group theory the concept of generic behavior is due to Gromov. His 
inspirational works [27l [28] turned the subject into an area of very active 
research, see for example, [2l[3l[2[Il[71[8l[ia[Il[l5l[l6l[I71[8l[9l[ini[32l 
[Ml[Ml[Ml[37l[Ml[Ml[l2l[50l[5ll[53l[59l[6l]. It turned out that the generic 
objects usually have much simpler structure, while the exceptions provide 
most of the difficulties. For instance, generic finitely generated groups are 
hyperbolic [271 151], generic subgroups of hyperbolic groups are free [26] . 
generic cyclically reduced elements in free groups are of minimal length in 
their automorphic orbits [39], generic automorphisms of a free group are 
strongly irreducible [52], etc. 

In practice, the generic-case analysis of algorithms is usually more im- 
portant than the worst-case one. For example, knowing generic properties of 
objects one can often design simple algorithms that work very fast on generic 
elements. In cryptography, many successful attacks exploit the generic prop- 
erties of random elements from a particular class, ignoring the existing hard 
instances |471 H8l B9l . In the precise form the generic complexity of algo- 
rithmic problems appeared first in the papers [Ml [351 (HI [E]- We refer the 
reader to a comprehensive survey |25j on generic complexity of algorithms. 

In this paper we lay down some techniques that allow one to measure 
sets which appear naturally when computing with infinite finitely presented 
groups. Our main idea is to approximate a given set by some regular subsets 
and estimate the asymptotic sizes of the regular sets using powerful tools 
of random walks on graphs and generating functions. The particular appli- 
cations we have in mind concern with the generic complexity of the Word 
and Conjugacy problems in free products with amalgamation and HNN ex- 
tensions. In general, such problems can be extremely hard. In [45] Miller 
described a free product of free groups with finitely generated amalgamation 
where the Conjugacy problem is undecidable; while in [l5] he gave similar 
examples in the class of generalized HNN-extension of free groups. However, 
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it has been proven in |15^ I16j that on a precisely described set RP of "reg- 
ular elements" in amalgamated free products and HNN extensions G the 
Conjugacy problem is decidable (under some natural conditions on the fac- 
tors), furthermore, it is decidable in polynomial time. Namely, it was shown 
in [121 US] that the group G (satisfying some natural assumptions) can be 
stratified into two parts with respect to the "hardness" of the conjugacy 
problem: 

• the Regular Part RP consists of so-called regular elements for which 
the conjugacy problem is decidable in polynomial time by the standard 
algorithms (described in [441 143j ). Moreover, one can decide whether 
or not a given element is regular in G; 

• the Black Hole BH (the complement of RP in G) consists of elements 
in G for which either the standard algorithms do not work at all, or 
they are slow, or the situation is not quite clear yet. 

The missing piece is to show that the set RP is, indeed, generic in G. 
This is not easy, the complete proof, which will appear in [23], relies on the 
techniques developed in the present paper. Now, a few words on the struc- 
ture of the paper. In Section [21 following [13], we describe some techniques 
for measuring subsets in a free group F, the asymptotic classification of 
large and small sets, and approximations via context-free and regular sets. 

In Section [3] we study, using graph techniques, Shreier system of repre- 
sentatives (transversal) of a finitely generated subgroup C in a free group 
F of finite rank. If 5 is a fixed Schreier transversal of C then s £ S is 
called stable (on the right) if sc £ S for any c £ C. Intuitively, the stable 
representatives are "regular", they are easy to deal with. 

In Section 131 we estimate the sizes of various subsets of F. In particular, 
we show that S is regular and thick (see definitions in Section [2]) , meanwhile 
the set Snst of non-stable representatives from S is exponentially negligible. 
Furthermore, the set Snst is exponentially negligible even relative to the set 
S. Our approach here is to "approximate" the sets in question by regular 
sets and to measure sizes of the regular sets using tools of random walks on 
graphs and Perron-Frobenius techniques. 

In Section [5] we develop a technique to compare sizes of different regular 
sets at "infinity" and give an asymptotic classification of regular subsets of 
F relative to a fixed prefix-closed regular subset L <^ F. The main result 
describes when regular subsets of L are "large" or "small" at infinity in 
comparison to L. Notice, in the case when L = F, this result has been 
proven in [13] (Theorem 3.2). 
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2 Preliminaries 



In this section, following |T^, we describe some techniques for measuring 
subsets in a free group -F, the asymptotic classification of large and small 
sets, and approximations via context-free and regular sets. 

2.1 Asymptotic densities 

Let F = F{X) be a free group with basis X = {xi, . . . 
notation throughout the paper. 

Let i? be a subset of the free group F and = {w ^ F 
sphere of radius k va. F. The fraction 

im = ^ 

Pfcl 

is the frequency of elements from R among the words of length k in F. The 
asymptotic density p{R) of R is defined by 

p{R) = limsup/fe(i?). 

fc— >oo 

R is called generic if p{R) = 1, and negligible if p{R) = 0. If, in addition, 
there exists a positive constant 5 < 1 such that 

l-b'' < fk{R) < 1 

for all sufficiently large k then R is called exponentially generic. Meanwhile, 
if fk{R) < 5^ for large enough k then R is exponentially negligible. In both 
the cases we refer to 5 as a rate upper bound. Sometimes such sets are also 
called strongly generic or strongly negligible, but we refrain from this. 
The Cesaro limit 

p'{R) = hm - (/i + • • • + /„) . (1) 

gives another asymptotic characteristic, called Cesaro density, or asymptotic 
average density. Sometimes, it is more sensitive then standard asymptotic 
density p (see, for example, [H], [60]). However, if limfc^oo /fc(-R) exists 
(hence is equal to p{R)) then p^{R) also exists and p^{R) = p{R). We will 
have to say more about p^{R) below. 

Asymptotic density gives the first coarse classification of large (small) 
subsets: 



. We use this 
|w| = /c } the 
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Coarse classification 

1) Generic sets; 

2) Visible or thick sets: the set R is visible if p{R) > 0; 

3) Negligible sets. 

Unfortunately, this classification is very coarse, it does not distinguish many 
sets which, intuitively, have different sizes. 

All our results in this paper concern with the strong version of the asymp- 
totic density p, when the actual limit limfc__»oo fk{R) exists. This allows one 
to differentiate sets with the same asymptotic density with respect to their 
growth rates. Thus generic sets R divide into subclasses of exponential, 
subexponential, superpolynomial, polynomial generic sets, with respect to 
the convergence rates of their frequency sequences {fk{R)}keN- The same 
holds for negligible sets as well. 



2.2 Generating random elements and multiplicative mea- 
sures 

One can use a no-return random walk Ws (s G (0, 1]) on the Cayley graph 
C{F,X) of F with respect to the generating set X, as a random generator 
of elements of F (see [H]). We start at the identity element 1 and either do 
nothing with probability s (and return value 1 as the output of our random 
word generator), or move to one of the 2m adjacent vertices with equal 
probabilities (1 — s)/2m. If we are at a vertex v ^ 1, we either stop at v 
with probability s (and return the value v as the output), or move, with 
probability 2^^, to one of the 2m — 1 adjacent vertices lying away from 
1, thus producing a new freely reduced word vxf^. Since the Cayley graph 
C{F,X) is a tree and we never return to the word we have already visited, 
it is easy to see that the probability ps{w) for our process to terminate at a 
word w is given by the formula 

and 

/is(l) = s. (3) 

For i? C F its measure fJ.s{R) is defined by fJ.s{R) = YlweB.l^si'w). Re- 
calculating fJ.s{R) in terms of s, one gets 

00 

k 



fls{R) = S^/fe(l - S 



k=0 
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and the series on the right hand side is convergent for ah s G (0,1). The 
ensemble of distributions {ns} can be encoded in a single function 



fi{R) : s G (0,1) ^ fis{R) G 

The argument above shows that for every subset R <^ F, fi{R) is an analytic 
function of s. 

It has been shown in [3] that fi{R) contains a lot of information about 
the asymptotic behaviour of the set R. To see where this information comes 
from renormalise the measures fig and consider the parametric family //* = 
{/Li*} of adjusted measures 

^^:H = (^^■-)■^^s{w). (4) 



2m -1 

This new measure /x* is multiplicative in the sense that 

H*{uov) = nl{u)nl{v), (5) 

where u o v denotes the product of non-empty words u and v such that 
\uv\ = \u\ + \v\ (no cancelation in the product uv). Moreover, if we denote 

then 

/.:(u;) = tH (7) 

for every non-empty word w. Assume now, for the sake of minor technical 
convenience, that R does not contain the identity element 1. It is easy to 
see that 

oo 

^^l{R) = Y,MR)t'' 

is the generating function of the spherical growth sequence 

rikiR) = \RnSk\ 

of the set R in variable t which is convergent for each t G [0, 1). 

The distribution has the uncomfortably big standard deviation a = 
, which reflects the fact that Hs is strongly skewed towards "short" 
elements. The mean length of words in F distributed according to Hs is 
equal to = ^ — 1, so — > oo when s — > 0. This shows that the 
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asymptotic behaviour of the set R at "infinity" (when Lg — > oo) depends on 
the behaviour of the function fi{R) when s — > O"*". 

Following [13] , for a subset R oi F we define a numerical characteristic 

oo 

MR) = lini KR) = lim s ■ V A(l - s)^ 

If n{R) can be expanded as a convergent power series in s at s = (and 
hence in some neighborhood of s = 0): 

fi{R) = mo + mis + m2S^ H , 

then 

//o(-R) = lim Ais(-R) = mo, 

and an easy corollary from a theorem by Hardy and Littlewood \30\ Theo- 
rem 94] asserts that I-Lq{R) is precisely the Cesaro limit p^{R)- 

A subset i? C F is called smooth [H] if f^{R) can be expanded as a 
convergent power series in s at s = 0. 

2.3 The frequency measure 

In this section we discuss the frequency measure, introduced in |14j . 

Let Wq be the no-return non-stop random walk on the Cayley graph 
C{F,X) of F (like Wg with s = 0), where the walker moves from a given 
vertex to any adjacent vertex away from the initial vertex 1 with equal 
probabilities l/2m. In this event, the probability X{w) that the walker hits 
an element w € F in \w\ steps (which is the same as the probability that 
the walker ever hits w) is equal to 

^(-) = 2m(2m-l)H-i' " ^ 1' ^(1) = 

This gives rise to a measure called the frequency measure on F, or 
Boltzmann distribution, defined for subsets i? ^ -F by 

A(i?) = ^A(^«), 

if the sum above is finite. One can view A(i?) as the cumulative frequency 
of R since 

oo 

XiR) = Y,fkiR)- 

k=0 
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This measure is not probabilistic, since, for instance, X{F) = oo, moreover, 
A is additive, but not cr-additive. 

A subset i? C F is called X-measurable, or simply measurable (since we 
do not consider any other measures in this paper) if X{R) < oo. Every 
measurable set is negligible. 

Linear approximation. If the set R is smooth then the linear term in 
the expansion of fi{R) gives a linear approximation of /^(-R): 

= TTT-o + mis + O(s^). 

In this case, tuq = fio{R) is the Cesaro density of R. An easy corollary of 
[30\ Theorem 94] shows that if ^o(^) = then 

oo 

mi = ^h{R) = X{R). 

k=l 

On the other hand, even without assumption that R is smooth, if R is 
measurable, then 

^o(-R) = and = lim = X(R). 

s-*0+ S 

2.4 Asymptotic classification of subsets 

In this section we describe a classification of subsets R in F, according to 
the asymptotic behavior of the functions f^{R)- 

Recall, that the function fJ,{R) is analytic on (0, 1) for every subset R of 
F. R is smooth if i-t{R) can be analytically extended to a neighborhood of 
0. The subset R is called rational, algebraic, etc, with respect to /U if the 
function ^{R) is rational, algebraic, etc. 

Asymptotic classification of sets. The following subtler classification 
of sets in F (based on the linear approximation of /u(i2)) was introduced in 

• Thick subsets: fio{R) exists, Ho{R) > and 

fj.{R) = fJ.Q{R) + ao{s), where lim ao(s) = 0. 

• Negligible subsets of intermediate density: fio{R) = but fJ-i{R) does 
not exist. 
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• Sparse negligible subsets: fJ.o{R) = 0, fJ-i{R) exists and 

= ^i{R)s + ai{s) where lim ^^^^^ = g. 

s^0+ S 

• Exponentially negligible sets: 

• Singular sets: /-fo(-R) does not exist. 

For sparse sets, the values of fii provide a further and more subtle dis- 
crimination by size. 

Lemma 2.1. A subset is sparse in F if and only if it is measurable. 

2.5 Context-free and regular languages as a measuring tool 

The simple observation in Section [2.21 that fJ,{R) is the generating function 
of the grows sequence {nk{R)}keN allows one to apply a well established 
machinery of generating functions of regular and context-free languages to 
estimate asymptotic sizes of subsets R in F. We refer to [31] on regular and 
context-free languages, and to [21] on regular languages in groups. 

Algebraic sets and context free languages. If the set R is an (unam- 
biguous) context free language then, by a classical theorem of Chomsky and 
Schutzenberger [19], the generating function /U*(i?) = '^nk{R)t^ , and hence 
the function ^{R), are algebraic functions of s. Moreover, if R is regular 
then n{R) is a rational function with rational coefficients \22\ 157]. 

It is well known that singular points of an algebraic function are either 
poles or branching points. Since /i(-R) is bounded for s £ (0, 1), this means 
that, for a context-free set R, the function fJ,{R) has no singularity at or 
has a branching point at 0. A standard result on analytic functions allows 
us to expand Hs{R) fractional power series: 

^is{R) = mo + mis^/" + mas^/" + • • • , 

n being the branching index. This technique was used in |16[ [T7] for numer- 
ical estimates of generic complexity of algorithms. 

If R is regular, than we actually have the usual power series expansion: 

fis{R) = + mis + 77125^ H ; 

in particular, /u(i?) can be analytically extended in the neighborhood of 0, 
so R is smooth. 

The following gives an asymptotic classification of regular subsets of F. 
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Theorem 2.2. |g 

1 ) Every negligible regular subset of F is strongly negligible. 

2) A regular subset of F is thick if and only if its prefix closure contains 
a cone. 

3) Every regular subset of F is either thick or strongly negligible. 

3 Schreier Systems of Representatives 
3.1 Subgroup and coset graphs 

In this section for a given finitely generated subgroup of a free group we 
discuss its subgroup and coset graphs. 

Let F = F{X) be a free group with basis X = {xi, . . . ,Xn}. We identify 
elements of F with reduced words in the alphabet XUX~^. Fix a subgroup 
C = {hi, . . . , hm) of F generated by finitely many elements hi, ... , hm G F. 

Following [33], we associate with C two graphs: the subgroup graph V = 
Tc and the coset graph r° = F^. We freely use definitions and results from 
|33| in the rest of the paper. 

Recall, that F is a finite connected digraph with edges labeled by el- 
ements from X and a distinguished vertex (based-point) 1, satisfying the 
following two conditions. Firstly, F is folded, i.e., there are no two edges 
in F with the same label and having the same initial or terminal vertices. 
Secondly, F accepts precisely the reduced words in X U X^^ that belong 
to C . To explain the latter observe, that walking along a path p in F one 
can read a word l{p) in the alphabet X U X~^ , the label of p, (reading x in 
passing an edge e with label x along the orientation of e, and reading in 
the opposite direction). We say that F accepts a word w iiw = ^{p) for some 
closed path p that starts at 1 and has no backtracking. One can describe F 
as a deterministic finite state automata with 1 as the unique starting and 
accepting state. 

For example, the graph F for the subgroup generated by xiX2x]~^ is 
shown in the picture below. 
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Given the generators hi, ... , hm of the subgroup C, as words from F[X), 
one can effectively construct the graph F in time 0(nlog* n) [?]. 

The coset graph (also known as the Schreier graph) T* = of C is a 
connected labeled digraph with the set {Cu | u S F} of the right cosets of 
C in as the vertex set, and such that there is an edge from Cu to Cv with 
a label x G X if and only if Cux = Cv. One can describe the coset graph 
F* as obtained from F by the following procedure. Let u G F and x ^ X 
such that there is no outgoing or incoming edge at v labeled by x. We call 
such V a boundary vertex of F and denote the set of such vertices by dT. For 
every such v £ dV and x £ X we attach to v a new edge e (correspondingly, 
either outgoing or incoming) labeled x with a new terminal vertex u (not in 
F). Such vertices u are called frontier vertices, we denote the set of frontier 
vertices of F by d~^r. Then we attach to u the Cayley graph C{F,X) of F 
relative to X (identifying u with the vertex 1 of C{F, X)), and then we fold 
the edge e with the corresponding edge in C{F,X) (that is labeled x and 
is incoming to u). Observe, that for every vertex v £T* and every reduced 
word w m X \J X~^ there is a unique path F* that starts at v and has the 
label w. By pyj we denote such a path that starts at 1, and by the end 
vertex oi pw Here is the fragment of the graph F* for C = (^xiX2x'^^y. 









x^ 


1 x^ ^ . x\ 








\ Ol 

X2 

X2 
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Lemma 3.1. F^ is the coset graph of C in F. 

Proof. See, for example, [33] . □ 

Notice that F = F* if and only if the subgroup C has finite index in F. 
Indeed, F = F* if and only if for every vertex f of F and every label x £ X, 
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there is an edge in T labeled by x which exits from v, and an edge with label 
X which enters v, but this is precisely the characterization of subgroups of 
finite index in F |33l Proposition 8.3]. 

A spanning subtree T of F with the root at the vertex 1 is called geodesic 
if for every vertex v £ V(T) the unique path in T from 1 to f is a geodesic 
path in T. For a given graph F one can effectively construct a geodesic 
spanning subtree T (see, for example, |33j). 

From now on we fix an arbitrary spanning subtree T of F. It is easy to 
see that the tree T uniquely extends to a spanning subtree T* of F* . 

Let V{T*) be the set of vertices of F*. For a subset Y C V{T*) and a 
subgraph A of F*, we define the language accepted by A and Y as the set 
L(A,y, 1) of the labels £{p) of paths p in A that start at 1 and end at one 
of the vertices in Y, and have no backtracking. Notice that the words i{p) 
are reduced since the graph F* is folded. Notice, that F = L(F*, ^(F*), 1) 
andC = L(F,{l},l) =L(F*,{1},1). 

Sometimes we will refer to a set of right (left) representatives of C as 
the right (left) transversal of C. Furthermore, to simplify terminology, a 
transversal will usually mean a right transversal, if not said otherwise. Re- 
call, that a transversal of C is termed Schreier if every initial segment of 
a representative from S belongs to S. 

Proposition 3.2. Let C be a finitely generated subgroup of F. Then: 

1) for every spanning subtree T of F the set St* = L(T* ,V(T*),1) is a 
Schreier transversal of C . 

2) for every Schreier transversal of C there exists a spanning subtree 
T of F such that S = St* ■ 

Proof. The statement 1) follows directly from Lemma [3. 11 To prove 2) notice 
that every reduced path p £ T* can be decomposed as p = pint °Pout, where 
Pint is a maximal reduced path in F, and pout is the tail of p outside of F. 
This decomposition is unique. Moreover: 

• a V € V{T) and p is a reduced path from 1 to f in F* then p passes 
only through vertices of F; 

• ii V £ V{T*) \ V{T) and p' and p" are two paths from 1 to v, where 
P' = P[nt ° P'ouv and p" = o then p'^^^ = p'^^^. 

Let 5 be a Schreier transversal of C in F and s £ S. Suppose that the 
reduced path ps ends at some vertex Vg in F. Then the whole path ps lies in 
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r. Let T be a subgraph of T generated by the union of all paths ps, where 
s £ S and Vg £T. Since /S is a Schreier transversal T is a maximal subtree 
of r. It is clear that S = St*- Hence, the result. □ 

Proposition 13.21 allows one to identify elements from a given Schreier 
transversal S" of C with the vertices of the graph T* , provided a maximal 
subtree of T is fixed. We use this frequently in the sequel. 

Corollary 3.3. The number of distinct Schreier transversals of C in F is 
finite and equal to the number of spanning subtrees of Tc- 

3.2 Schreier transversals 

In this section we introduce various types of representatives of C in F relative 
to a fix basis X of F. 

Definition 3.4. Let S* be a transversal of C. 

• A representative s £ S is called internal if the path ps ends in F, i.e., 
Vs G By Sint we denote the set of all internal representatives in 
S. Elements from 5cxt = ^ \ ^int are called the external representatives 
in S. 

• A representative s G 5" is called geodesic if it has minimal possible 
length in its coset Cs. The transversal S is geodesic if every s E 5 is 
geodesic. 

• A representative s G S is called singular if it belongs to the generalized 
normalizer of C: 

NUC) = {f GF\f-'cfnc^i}. 

All other representatives from S are called regular. By 5'sin and, respec- 
tively, 5reg we denote the sets of singular and regular representatives 
from S. 

• A representative s £ S is called stable (on the right) ii so £ S for any 
c S C. By Sst we denote the set of all stable representatives in S, and 
Suns = S ^ Sst is the set of all non-stable representatives from S. 

In the following lemma we collect some basic properties of various types 
of representatives. Recall that the cone defined by (or based at) an element 
u £ F is the set C{u) of all reduced words in F that have u as an initial 
segment. For u,v £ F we write uov ii there is no cancelation in the product 
uv, i.e., \uv\ = \u\ + \v\. In this case C{u) = {w £ F \ w = u o v,v £ F}. 
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Proposition 3.5. Let S he a Schreier transversal for C, so S = St* for 
some spanning subtree T* of T* . Then the fohowing hold: 

1) S'int is a basis of C, in particular, |S'int| = |^(r)|. 

2) Se^t is the union of finitely many coni C{u), where Vu G d~^T. 

3) Ssin is contained in a finite union of double cosets CsiS2^C of C, where 

Si,S2 e S'int- 

4) S'uns is a finite union of left cosets of C of the type siSg ^C, where 

Sl,S2 G S'int- 

Proof. 1) is well-known, see [12], for example. 2) follows immediately from 
the construction. To see 3) notice first that Sgm C Np{C) and Np{C) is the 
union of finitely many double cosets CsC, where s £ Sgin, and furthermore, 
every such coset has the form CsC = CsiSg^C, where si,S2 G Smt (see 
Lemma 5 in [12] . or Propositions 9.8 and 9.11 in [33], or Theorem 2 in [32]). 

To see 4) assume that s € S is not stable, so there exists an element 
c £ C such that sc S. Then s = si o t,c = t^^ o d, sc = si o d. We claim, 
that the terminal vertex of si lies in F (viewing s as a path in F*). Indeed, 
if not, then si, as well as si o d, is in ^ext - contradiction. Hence, si G S'int- 
Since c = o d there is a closed path in F with the label o d, starting 
at 1(7- Let S2 G S'int be the representative of t^^. Then S2d = ci £ C, hence 
sc = si o d = S1S2 ^ci, so s G S1S2 ^C, as claimed. 

□ 

Proposition 3.6. Let S" be a Schreier transversal for C, so S' = St* for 
some spanning subtree T* of F*. Then the following hold: 

1) If T* is a geodesic subtree of F* (and hence T is a geodesic subtree of 
F) then S is a geodesic transversal. 

2) If C is a malnormal subgroup of F then S'sin = 0. 

3) S'sin ^ Suns- 

Proof. 1) is straightforward (see also [33]). 

2) If C is malnormal then Np{C) = 1, so S'sin = 0- 

3) If s G 5sin then c = s~^cis for some non-trivial c, ci G C, so cis = sc. 
Since sc / s we conclude that sc ^ S, hence s G S'uns- D 
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4 Measuring subsets of F 



Recall that a finite automaton ^ is a finite labeled oriented graph (possibly 
with multiple edges and loops). We refer to its vertexes as states. Some of 
the states are called initial states, some accept or final states. We assume 
further that every edge of the graph is labeled by one of the symbols x^^, x G 
X, where F = F{X) is a free group of finite rank m. The language accepted 
by an automaton A is the set L = L(A) of labels on paths from initial to 
accept states. An automaton is said to be deterministic if, for any state 
there is at most one arrow with a given label exiting the state. A regular 
set is a language accepted by a finite deterministic automaton. 

The following facts about regular sets are well known. Let A and B are 
regular subsets in F. Then: 

• the sets AU B, AD B and A \ B are regular. 

• The prefix closure ^ of a regular set A is regular. Here, the prefix 
closure A is the set of all initial segments of all words in A. 

• ab = a ob for any a £ A,b £ B then AB is regular. 

• ab = a ob for any a,b £ A then A* is regular. 
The following notation is useful. For u,v £ F define 

c{u,v) = ^(l^l + 1^1 ~ l'"''^!) 
- the amount of cancelation in the product uv. 

Proposition 4.1. Let Ri and R2 be subsets of F. Then the following 
statements hold: 

1) If i?i C i?2 and i?2 is negligible (exponentially negligible) then so is 
Ri. 

2) If i?i,i?2 are negligible (exponentially negligible) then so is RiU R2. 

3) If i?i and R2 are negligible (exponentially negligible) then so is the set 

Rio R2 = {rir2 \ ri £ Ri, c{ri,r2) = 0}. 

4) If i?i and R2 are negligible (exponentially negligible) then so is the set 

RioR2 = {rir2 \ r^ £ Ri, c{ri,r2) < t}. 
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Proof. The proof is straightforward. □ 

Definition 4.2. Let Ri and R2 be subsets of F and f : Ri ^ R2 & map. 
Then: 

• / is called d-isometry, where d is a non-negative real number, if for 

any w £ Ri 

\w\-d< I/HI < |w;| +d. 

• / has uniformly bounded fibers if there exists a constant c such that 
every element w £ R2 has at most c pre-images in Ri. 

Proposition 4.3. Let Ri and R2 be subsets of F. Then the following 
statements hold: 

1) If / : i?i ^ i?2 is a surjective d-isometry and Ri is negligible (expo- 
nentially negligible) then so is i?2. 

2) If / : i?i ^ R2 is a d-isometry with uniformly bounded fibers and R2 
is negligible (exponentially negligible) then so is 

Proof. Notice that for k > d 

k+d 
j=k-d 

and 1) follows. Similarly, 

k+d 

fk{Ri) < c fj{R2) 

j=k-d 

for k > d and 2) follows. □ 

Proposition 4.4. Let C be a finitely generated subgroup of infinite index 
of a free group F. Then every Schreier transversal of C in F is regular and 
thick. 

Proof. By definition S = Si^t U 5cxt • By Proposition 13.51 the set Si^t is finite 
and the set Sext is a non-empty finite union of cones. By Theorem 12.21 each 
cone in S'cxt is thick. Therefore, the set Sexti as well as the set S, is thick. 
Clearly, every cone is regular, so is the set S. □ 

Proposition 4.5. Let C be a finitely generated subgroup of infinite index 
in F. Then the following hold: 
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1) C is exponentially negligible in F and one can find some upper bound 
6 < 1 for the growth rate of C. 

2) Every coset of C in F is exponentially negligible in F. 

Proof. 1) follows from the Proposition 1 and Corollary 1 in |6j. 

2) follows from 1) above and 4) from Proposition 14.11 □ 

Proposition 4.6. Let C be a finitely generated subgroup of infinite index 
in F. Then the following hold: 

1) (7* = y is exponentially negligible in F. 

2) For every c G C the set conjugacy class = {f~^cf\f G F} is 
exponentially negligible in F. 

Proof. The statement 1) has been shown in |14| and also in Proposition 1.10 
in [5]. The statement 2) is shown in Proposition 1.11 in [5] □ 

Proposition 4.7. Let C be a finitely generated subgroup of infinite index 
in F and 5 is a Schreier transversal of C in F. If 5o ^ 5" is a exponentially 

negligible subset of F then the set |J Cs is exponentially negligible in F. 

seSo 

Proof. By Proposition 13.51 5* = Si^t U Scxt, where S'int is a finite set and Scxt 
is a union of finitely many cones C{u),u G d^T. It suffices to prove the 
result for Sq H C(n) for a fixed u G d~^T. To this end we may assume from 
the beginning that C C(u). If s is the representative of u in S then every 
word from C{u) contains s as an initial segment. Since s is not readable in 
r the amount of cancelation c{w, t) in the product wt, where w G C and 
t G C(n) does not exceed the length of s. Hence 

CSq = C o Sq 

and the result follows from the statement 4) of Proposition 14.11 □ 

Proposition 4.8. Let A and B be finitely generated subgroups of infinite 
index in F. Then for any w £ F the double coset AwB is exponentially 
negligible in F. 

Proof. Observe, that ^ifi? = ^-B"" ^■w, so by the statement 2 of Proposition 
14.51 it suffices to show that AB^ is exponentially negligible. Since B^ is 
just another finitely generated subgroup of infinite index in F one can assume 
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from the beginning that w = 1. Let 5 be a geodesic Schreier transversal for 
A in F. Then 

AB= \J As 

seSo 

for some subset Sq C S. By Proposition 14.71 it suffices to show that the 
subset Sq is exponentiahy negligible. Since the set 5int is finite we may 
assume that Sq C 5cxt- Now we construct an r-isometry a : Sq ^ B. Let 
Ta be the spanning subtree of Ta such that S = S^j^* and Tg be a spanning 
geodesic subtree of F^. Denote by d the maximum of the diameters of 
the trees Ta and Tb- To describe the map a choose an arbitrary element 
s £ Sq. Without loss of generality assume that \s\ > d, because there are 
only finitely many such s that have smaller length and by Proposition 14.51 
they will not extremely change asymptotic size of AB since A of infinite 
index in F. Then as = h for some a £ A and b £ B. We claim that there 
exists an element hg £ B such that js^J"^] < 2d. Indeed, the cancelation in 
the product as is at most d (see the argument in Proposition 14. 7p . Hence s 
and h have a common terminal segment t of length at least |s| — d (recall that 
|s| > d). It follows that in the graph Tb there exists a path from some vertex 
V to 1b with the label t^. Then bg = t^t G B and = |st~^t^"^| < 2d. 

Hence s and bg has a long common terminal segment and differ only on the 
initial segment of length at most 2d. It follows that the map a : s — > 6s 
gives a 2d-isometry a : Sq ^ B. Notice that a has uniformly bounded 
fibers. Indeed, if a{si) = a{s2) = b then si and S2 differ from b, hence from 
each other, only on the initial segment of length at most 2d. So there are 
at most (2(i)^l^l such distinct elements. Since B is exponentially negligible 
by Proposition 14.31 the set Sq is also exponentially negligible, as claimed. 
This proves the result. Notice, that the property being geodesic for Schreier 
transversal S for A in F is not crucial for our prove. Namely, for arbitrary 
Shreier transversal S all conclusions can be repeated with slightly different 
constant. □ 

Now we can state the main result of the section. 

Theorem 4.9. Let C be a finitely generated subgroup of infinite index in F 
and S a Schreier transversal for C . Then the following hold: 

1) The generalized normalizer Np{C) ofC in F is exponentially negligible 
in F. 

2) The set of singular representatives Ssin is exponentially negligible in 
F. 
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3) The set Suns of unstable representatives is exponentially negligible in 
F. 

Proof. To see 1) recall that the generalized normalizer Np{C) of C in F is 
a finite union of double cosets of C in F. Therefore Np[C) is exponentially 
negligible in F by Proposition I4.8[ 
2) follows immediately from 1). 

To prove 3) observe that S^^ns is a finite union of left cosets of C (see 3) 
in Proposition 13. Sp . Now the result follows from Proposition 14.51 □ 

Theorem 14.91 can be strengthen as follows. 

Corollary 4.10. Let C be a finitely generated subgroup of infinite index in 
F. Then the sets 

Sin{C) = IJ Ssin, U ns{C) = |J 5uns, 
s s 

where S runs over all Schreier transversals of C, are exponentially negligible. 

Proof. By Corollarv 13.31 there are only finitely many Schreier transversals of 
C. Now the result follows from Theorem 14.91 and Proposition 14.11 □ 

5 Comparing sets at infinity 

5.1 Comparing Schreier representatives 

In this section we give another version of Theorem 14.91 To explain we need 
a few definitions. 

For subsets R,L of F we define their size ratio at length k by 

fkiR) \RnS,\ 

The size ratio p{R, L) at infinity of R relative to L (or the relative asymptotic 
density) is defined by 

p{R, L) =limsup fk{R,L). 

k~*OD 

By riiR) we denote the cumulative size ratio of R relative to L: 



rL{R) = Y.fk{R.L). 



k=l 
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We say that R is L-measurable, if ri{R) is finite. R is called negligible 
relative to L if L) = 0. Obviously, an L-measurable set is L-negligible. 
A set R is termed exponentially negligible relative to L (or exponentially 
L-negligible) if fk{R,L) < for all sufficiently large k. 
The following result is simple, but useful. 

Proposition 5.1. Let R be an exponentially negligible set in F. 

1) For any w £ F the set i? is a exponentially negligible relative to the 
cone C{w). 

2) The set R is exponentially negligible relative to any exponentially 
generic subset T of F. 

Proof. Observe, that fk{C{w)) = l/2m(2m - l)!"-!"! is a constant. Since 

fk{R) 



fk{R,C{w)) 



fk{C{w)) 



it follows that R is exponentially negligible relative to C{w). This proves 
!)• 

To prove 2) denote by p and q the corresponding rate bounds for R and 
T, so fk{R) < p'', fk{T) > 1 — q'' for sufficiently large k. Then, for such k, 

fk{R) ( P \^ 

fk{R,T) = < r = 

fk{T) l-q^ \{i_qk)\j 

Since 

Lim = p 



it follows that for any e > 

/fc(i?,r)<(p + e)'= 
for sufficiently large k, as claimed. 



□ 



Corollary 5.2. Let C be a finitely generated subgroup of infinite index in 
F and S a Schreier transversal for C. Then the following hold: 

1) The set of singular representatives S'sin is exponentially negligible in 
S. 
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2) The set S'uns of unstable representatives is exponentially negligible in 
S. 

Proof. The statements of this corollary follow immediately from Theorem 
K9\ and Propositions [331 [Ml and EH □ 

5.2 Comparing regular sets 

In this section we give an asymptotic classification of regular subsets of F 
relative to a fixed prefix-closed regular subset L C F. 

For this purpose we are going to describe how one can use a random 
walk on the finite automaton B recognizing regular subset R (1 L similar to 
the one in Section [2.31 It will be convenient to further put B to special form 
consistent to L. 

Recall Myhill-Nerode's theorem on regular languages (see, for example, 
|21j . Theorem 1.2.9.) For a language R over an alphabet A consider an 
equivalence relation on A* defined as follows: two strings wi and W2 are 
equivalent if and only if for each string u over A the words wiu and W2U 
are either simultaneously in R or not in R. Then R is regular if and only if 
there are only finitely many ~-equivalence classes. 

Now, let R L. Define an equivalence relation on L such that wi ~ W2 
if and only if for each u £ F the following condition holds: wiu = wi o u 
and wiu € R ii and only if W2U = W20 u and W2U G R. 

The following is an analog of Myhill-Nerode's theorem for free groups. 

Lemma 5.3. Let R L Q F and L prefix-closed and regular. Then R is 
regular if and only if there are only finitely many ^-equivalence classes in 
L. 

Proof. The proof is similar to the original one. We give a short sketch of 
the most interesting part of it. If the set of the equivalence classes is finite 
one can define an automaton B on the set of equivalence class as states. If 
X S X U and [w] is the equivalence class of some w such that wox ^ R 
then one connects the state [w] with an edge labeled by x to the state [wx] . 
The class [e], where e is the empty word, is the initial state, while a state 
[u;] is an accepting state if and only \i w ^ R. In this case L{B) = R. □ 

Since R is regular, we suppose that B as in Lemma 15.31 and modify it 
in the next way. Without loss of generality we can assume that A is in 
the normal form, i.e., it has only one initial state / and doesn't contain 
inaccessible states. 
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Pic. 3. Splitting the states of the automaton B. 

Let S = [w] be a state of B. Denote by S*^^ the uniquely defined state 
in A which is the terminal state of the path with the label w in A, starting 
at [e]. The state is well-defined, it does not depend on the choice of w. 
We call the prototype of S. 

Since B accepts only reduced words in X U one can transform B to 
a form where the following hold: 

a) B has only one initial state / and one accepting state Z. 

b) For any state S of B, all arrows which enter 5 have the same label 
X £ X U and arrows exiting from S cannot have label (this 
can be achieved by splitting the states of B, see Pic. 3). We shall say 
in this situation that S has type x. 

c) For every state S of B there is a direct path from S to the accept state 
Z. 

d) There are no arrows entering the initial state /. 

The final version of obtained automaton B we will call an automaton 
consistent with A. 

Now we are ready to define a no-return random walk on B as it was 
claimed above. Namely, let B be consistent with A and let 5" be a state in 
B. Denote by = vi^S^^) the number of edges exiting from the prototype 
state S'P'^ in A. The walker moves from 5 along some outgoing edge with 
the uniform probability i. In this event, the probability that the walker 
hits an element w ^ R va. \w\ steps (when starting at [e]) is the product of 
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frequencies of arrows in a direct path from the initial state / to the accept 
state Z with the label w. This gives rise to the measure A^, on i?: 

oo 

w&R k=0 

where 

f',{R,L)= ^l{w). 

Note that, generally speaking, f'^{R,L) differs from fk{R,L) defined in 
section 15.11 Indeed, walking in B we have different number of possibilities 
to continue our walk on the next step depending on way we chose. On the 
other hand, = fk{R,F{X)). 

Now we can use the tools of random walks to compute Xl{R)- Notice, 
that Al is multiplicative, i.e., 

\l{uv) = \l{u)\l{v) 

for any u,v G R such that uv = u o v and uv G R. We say that R 
is XL-measurable, if \l{R) is finite. A set R is termed exponentially XL- 
measurable) if f'j.{R,L) < for all sufficiently large k. 
The following result is simple, but useful. 

Let w G F. The set Cl{w) = L n C{w) is called an L-cone. Obviously, 
Cl{w) is a regular set. We say that Cl{w) is L-small, if it is exponentially 
A^-measurable. 

The following is the main result of this section. 

Theorem 5.4. Let R be a regular subset of a prefix-closed regular set L 
in a free group F. Then either the prefix closure R of R in L contains a 
non-small L—cone or R is exponentially XL-measurable. 

Before proving the theorem we establish a few preliminary facts. We fix 
a prefix-closed regular subset L of F. 

Proposition 5.5. Let Ri and R2 be subsets of F. Let also P be one of 
the properties { "to be L— measurable" , "to be exponentially L— negligible" , 
"to be Al— measurable" , "to be exponentially A^— measurable"}. Then the 
following hold: 

1) If i?i C R2 and R2 has property P then so is Ri. 

2) If Ri, R2 have property P then so is i?i U i?2- 
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3) If Ri and R2 have property P then so is the set 

Rio R2 = {rir2 I ri G Ri, c(ri,r2) = 0}. 

Proof. The proofs are easy. □ 

To strengthen the last statement in Proposition 15. 51 we need the following 
notation. For a subset T C F put = T and define recursively T^_^_i = 
T° o T. Denote 

00 

k=l 

Lemma 5.6. Let T he a regular set and a number < g < 1, such that 
f'^{T,L) < for every positive integer k. Then the set is exponentially 
Xf^— measurable. 

Proof. Every word lu G of length k comes in the form w = U1OU20. . .out, 
where ttj's are non-trivial elements from T and /c = + . . . + On the 
other hand, if /c = /ci + . . . + /cj is an arbitrary partition of k into a sum 
of positive integers and ui, . . . ,ut are words in T such that Ui = ki, then 
u) = ui...ut G T^. Since is multiplicative every partition of k adds 
to fi,{T;^,L) a number /^^ (T^, L) . . . /^T^, L), which is bounded from 
above by = If p[k) is the number of all partitions of k into a 

sum of positive integers then f'j^{T^,L) < p{k)q^. It is known (Hardy and 
Ramanujan) that 




Hence f'f.{T^,L) < q^, for some < q < qi < I and all sufficiently large k, 
so is exponentially A^— measurable, as claimed. □ 

Proof of Theorem \5.4\ In the most part we follow the proof of Theorem 
12.21 from [H] , Suppose that all L-cones in R are non-small. Since i? C i? 
by Proposition 15.51 we can assume that R itself is prefix-closed in L. We 
have to prove that R is exponentially A^-measurable. Let R = L{B) and B 
consistent to A (where A recognize L). 

It is convenient to further split B into two parts. Denote by Bi the 
automaton obtained from B by removing all arrows exiting from Z. 
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Pic. 4. An automaton B. 



Co) 



Pic. 5. An automaton Bi. 

Let B2 be the automaton formed by all states in B that are accessible 
from the state Z, with the same arrows between them as in B; Z is the only 
initial and accepting state of B2- 




Pic. 6. Automaton B 



2- 



We assign to arrows in Bi and B2 the same frequencies as to the corre- 
sponding arrows in B. If Ri and R2 are the languages accepted by Bi and 
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B2 then, obviously, R = Ri o R2. By Proposition 15.51 to prove the theorem 
it suffices to show that Ri and R2 are exponentiahy Ai-measurable. 

Claim. The set R2 is exponentially XL-measurable. 

Proof of the claim. Notice, that for every w (z Ri w o R2 CI L{A) = R and 
W0R2 is an L— cone. It is easy to see, that R2 is exponentially A^^-measurable 
if and only if so w o R2 is. 

Let R^ C R2 be the subset consisting of those non-trivial words w (z R2, 
whose paths pw visit the state Z of B2 only once. The set R3 is regular - it 
is accepted by an automaton 83, which is obtained from B2 by splitting the 
state Z into two separate states: the initial state Zi and an accepting state 
Z2, in such a way that the edges exiting from Z in B2 are now exiting from 
Zi and there no ingoing edges at Zi, while there are no edges exiting from 
Z2 and all those arrows incoming in Z in B2 are now incoming into Z2. 




It follows immediately from the construction, that 

R2 = {e} [j Rs \J{R3 o i?3) \JiR3 oR3oR3)\J... = (R^y^ 

so 

Xl{R2) = XiiRs) + {Xl{R3)? + (XiiRs)? + ■ • • ■ (8) 

By Proposition 15. 61 it suffices to show that there is a number q,0 < q < 1, 
such that f'j,{R3,L) < q'^ for every k (not only for sufficiently large k). It is 
not hard to see that this condition holds if R3 is exponentially A ^-measurable 
and Xl{R3) < 1) so it suffices to prove the latter two statements. 

By our assumption all L-cones in R = R are L— small. If for every 
state [w] = S in B2 and every x £ X U there is an outgoing edge 
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labeled by x at [w] if and only if the same holds for the state S"^^ in A 
(i.e., B2 is X— complete relative to A) then for every given w £ Ri one 
has C(t(;) Ci R = w o R2 = C{w) Ci L, so w o R2 is an L-cone. Hence it is 
L— small, i.e., exponentially A^-measurable, but then the set R2, hence R2, 
is exponentially Ai-measurable, as claimed. 

This implies that for some state S there are less then 1/ = v[S^''') arrows 
exiting from 5. Consider a finite Markov chain M with the same states as in 
B-i together with an additional dead state D. We set transition probabilities 
from Z2 to Z2 and from D to D being equal 1. Every arrow from a state 
S in B3 gives the corresponding transition from the state S va. A4 which 

we assign the transition probability — . If at some state S of type x in 

V 

there is no exiting arrow labeled y G {X U \ in M. we make a 

transition from S to D with the transition probability — . This describes A4. 

V 




Pic. 8. An automaton M. 

The states Z2 and D of Markov chain are absorbing, and all other 
states are transient. The probability distribution on concentrated at 
the initial state converges to the steady state P which is zero every- 
where with the exception of the two absorbing states Z2 and D. Obviously, 
P(Z2) = \l{Rz)- Since PiD) / we have \l{Rz) = -P(^2) < 1, so one 
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of the required conditions on holds (for more details on this proof we 
refer to [14^ I41|). The other one follows directly from Corollary 3.1.2 in 
|41j . which claims that in this case R3 is exponentially Ai-measurable. This 
proves the claim. 

A similar argument shows that Ri is exponentially A/,-measurable. This 
proves the theorem. 
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